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0.    Introduction 

In  [1]  we  developed  the  distribution  theory  necessary  to  carry  out  Bayesian 
analysis  of  the  multivariate  Normal  process  when  neither  the  mean  vector  nor  the 
variance-covariance  matrix  of  the  process  is  known  with  certainty.   The  class  of 
natural  conjugate  distributions  of  the  process  was  identified  and  defined  in 
formula  (6)  of  [1],  We  labelled  members  of  this  class  "Normal-Wishart"  and  used 
Normal-Wishart  priors  to  carry  out  prior  to  posterior  and  pre-posterior  analysis. 
In  section  1.3  of  [1]  we  pointed  out  that  the  Normal-Wishart  class  lacks  flexi- 
bility in  a  certain  sense. 

Furthermore,  Rothenburg  [4  ]  points  out  an  additional  difficulty  associated 
with  natural  conjugate  analysis  of  the  Multinormal  process  when  this  analysis  is 
applied  to  reduced  form  systems;  namely,  if  the  joint  prior  of  (n,  h) ,  the  matrix 
of  coefficients  of  the  system  and  the  inverse  of  the  variance-covariance  matrix  of 
the  residual  error  vector  respectively,  is  in  the  natural  conjugate  family,  then 
the  marginal  variances  of  all  coefficients  in  the  ith  row  of  n  are  proportional 
to  the  marginal  variances  of  coefficients  in  the  jth  row  of  H.   In  practical 
problems,  such  a  restriction  is  often  undesirable.   Zellner  [  3  ]  attempts  to  deal 
with  this  problem. 

Martin  [  3  ]  highlights  another  restrictive  feature  of  the  Normal-Wishart 

family  of  priors.   The  parameter  V  of  a  Normal-Wishart  prior  as  defined  in  (2) 

~-I 
below  is  (v-2)  times  the  expected  value  E(h   )  of  the  variance-covariance  matrix 


of  the  data  generating  process  (1).   He  goes  on  to  show  that  given  (V,  v)  and 
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letting  h  „  be  the  (a,  P)  element  of  the  (r  x  r)  matrix  h   ,  for  1  <  a,  P,  7,  &  <  r. 


^^'^^p)  =  3(v-2)}v-4)  [f^  V^p  +  V^  Vpp] 
and 

'°"^^a^  \l^   -   3(v-2)(v-4)  f W^  ^aP^5  -^  Vp5  "*■  ^a5^P7^  ' 
Thus  specification  of  v  and  V  completely  specifies  the  variance  of  all  elements  of 
the  variance-covariance  matrix  of  the  process  and  fixes  the  covariances  among 

elements  of  this  matrix  as  well.   This  is  generally  an  undesirable  state  of  affairs 

~-l 
as  one  may  reasonably  wish  to  set  the  mean  of  h   and  the  variances  and  covariances 

among  its  elements  in  a  way  that  does  not  jive  with  the  above  formulae.   In 
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particular,  note  that  the  (prior)  variance  of  the  variances  of  the  process  are, 
for  a=l,2,...,r,  restricted  to  be 

Var(h-S  =  —^ =  \   E^rS 

°^    (v-2)2(v-4)   ^-"^     "« 

Our  purpose  here  is  to  extend  the  Normal-Wishart  class  of  priors  so  as  to 

allow  much  greater  flexibility  in  the  assignment  of  priors  to  the  parameter  of  a 

multivariate  Normal  data  generating  process,  while  retaining  as  much  analytical 

simplicity  as  possible.   In  particular,  we  identify  an  extension  that  liberates 

us  from  the  tyranny  outlined  in  the  preceding  paragraph  and  allows  us  to  assign 

~-l 
(prior)  variances  much  more  freely  to  the  diagonal  elements  of  h  --the  marginal 

variances  of  elements  of  y^,  the  mean  vector  of  the  data  generating  process. 

Throughout  we  use  the  notation  and  conventions  adopted  in  [1]. 


1.1  Natural  Conjugate  Distributions  of  (ji,  h) ,  ^,   and  h 

We  define  an  r-dimensional  Independent  Multinormal  process  as  one  that 

~(1)     ~(i) 
generates  independent  r  x  1  random  vectors  x   ,...,x   y-   with  identical 


densities 

^N 


^i^\^\^>    h)  =  (2n)-*'^  e-2(2i-ii)'h(x-ti)  |h|i 

(1) 

-00  <  X  <  +eo  , 

-00  "^  U  "^  "^    J 

h  is  PDS 

When  both  ^   and  h  are  random  variables,  the  natural  conjugate  of  (1)  is  the 

(r) 
Normal-Wishart  distribution  fxV.  (tL;  ^\^>    Y^  ^)    v)  defined  as  equal  to 


k(r,  V)  e"^"(ii"2!)'^(ii-2)  |hp^  e-^^^  ^^   ^*  |y*|i(v+r-l)  |h|?v-l   .■ 

^N^^^^itllB.  hn)  f^^'^hjV,  V)    if  n  >  0,  V  >  0  ,  ^^a) 

0  otherwise 


where 


otherwise 


(2b) 


(2c) 


and 

k(r,  V)  =  2-ir(v+r)  „-ir(r+l)  Jrb    ^_^^  p(i(^+r-i) )  J'^   .        (2d) 

2.   Some  Extensions 

Many  possibilities  for  generalizing  (2)  exist.  We  shall  restrict  our  atten- 
tion here  to  generalizations  which  (i)  are  closed  under  the  binary  operation  of 
going  from  a  prior  to  a  posterior  distribution  of  (j^,  h)  in  that  both  prior  and 
posterior  distributions  have  the  same  functional  form,  (ii)  allow  us  to  easily 
derive  marginal  distributions  of  ^   and  of  h  and  (iii)  enable  us  to  do  preposterior 
analysis  in  a  simple  fashion. 

The  extensions  exploited  here  use  generalizations  of  the  Wishart  distribution 
due  to  Bellman  [  2  ]. 

To  foreshadow  the  effect  of  extending  the  natural  conjugate  family,  we 
tabulate  some  moments.   Let 

^   =  £  =  [a. . ]  ,   dim(r  x  r) 

h   =  £  =  {o-^-^,...  ,o^^,a^^,...,a^^)      ,  dim(l  x  r(r+l))  , 

h"^  =  £  =  E(£)  , 
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V(£)  =  E(a-o)  (o-o)*^   , 


vfti 


V(ii)     E(ii-m)(a-£) 


[E(^-m)  (£-£)'']   V(£) 

In  Table  I  we  compare  E(^),  a_,    V(£)  ,  and  v/  ^  \when  we  assign  a  Normal-Wishart 
prior  with  parameter  (m,  n,  V,  v)  to  (y^,    h)  with  these  same  moments  when  an 
extended  Type  1  Normal-Wishart  prior  with  parameter  (m,  n,  V^  v^) ,  v^  =  (Vi,...,v  ) 


<%/    '~^^ 


is  assigned  to  (j_i,  h)  .   Here  J  is  an  (r  x  r)  upper  triangular  matrix  such  that 
J  I  =  Y   ^  3  is  an  (r  X  r)  diagonal  matrix  whose  diagonal  elements  are  functions 
of  V  alone  and  are  later  shown  to  fulfill  the  recursion  relation' 


JJ 


J -^ ) -'}'''  )^ 


Elements  of  W(y,  v)  below  are  shown  by  Martin  to  be 


W,.(y,  V)  =  cov(a^p,  a^g)  =  3(^.,^(^.^)  [^^  ^aP V+Vp6+V^6^p,^  • 


W.  .(V,  V)  .  var(a^p)  =  3(,.,;(,.,)  [^  V^^  ^  ^^^^^^ 


■'•^J--'-^2,..,,r, 


and  we  show  that  when 


r=2,^ 


and 


^ll(|^  V)  =  var(aj_^)  = 


*I'33(T,  v) 


2(a2-2) 


4(a2-f)(a^-|) 


-1  4     -12   -1  ?  ^?? 


14  ^22  ^2"^. 


^11 


11  ^l^' 


-   E^(^22>  ' 


4, 


The  a-s  are  functions  of  v^  alone,  as  defined  in  (4c)  below. 

The  extension  of  the  results  shown  here  to  arbitrary  r  is  straight  forward, 


but  tedious,  and  will  be  presented  in  a  later  draft, 


where  t. .  denotes  the  iith  element  of  T 

LJ 

TABLE  1 


Norma 


1-Wishart~     Extended  Type  1  Normal -Wishart 


E(ii) 


m 


m 


E(a)  =  V(ia):       :^   V 


I   n  I 


-1 


V(a) 


W(V,  v) 


v-2 
0 


W(V,  V) 


^(T,  v) 
I'-'-Q  l^'^  0 

0        ^j^(T,  V) 


In  a  later  section  we  discuss  a  numerical  example  and  compare  the  above. 


2.1  Bellman's  Extension  of  an  Integral  Identity  of  Siegel 


In  [ ;l]  Bellman  generalizes  Siegel's  identity: 

r  (V)  |vri(v+r-l)  ^  Te-itr  h  Y  |h|iv-l^^  ^ 


V  is  (r  X  r)  PDS  , 

V  >  0   . 


(3) 


iMartin 


[3  ]. 
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and 


r^(v)  =  n'*''^''"^^  r(v)  r(v4)...r(v4(r-i))  . 


Letting  h  be  an  (r  x  r)  real  symmetric  matrix,  V   =(V^-),  1  <  i,  j  <  /, 
and  hr--|  =  (h.  .)  >  /  <  i^  J  <  r,  he  shows  in  effect  that 

-itr  h  V    ^v  -1  , 

r    ^  lh\      ""  ^     ir(r-l)  ,   P-^^^r+'^-^^r-l  ,  [j]  "^ 


J  r        V.  ^     =  r^-'  1=1  j=l 


(4a) 


where 


a  =  ^r(Vj.+r-l)  -  1^   (r-j+Dv^.^  ,      p  =  j|^  v^   ,  (4b) 


^r=  2(Vj.+r-l)   ,   a^  =  a^  -  ^eJ  Vj^   ,   j=l,2, . . .  ,r-l  , 


(4c) 
(4d) 


a  =  (aj^,a2,..o,a^)   ,  r^(a)  =  ^H^  r(aj-i(j-l)) 

and  the  v  must  be  such  that  a.  >  4(j-l) .  We  shall  call  the  integrand  in  (4)  the 
J  J 

kernel  of  a  Type  1  extended  Wishart  density  fAf  (h|v,  v) ,  with  parameter  (V,  v) 

where  v  =  (v,  , . . .  ,v  ) .   Henceforth  we  shall  let 
—     l-"    '    r 

r*(v)  =  2«  nu>^(^-l)  r(a^4(r-l))..,r(a24)  r(a^) 
for  notational  simplicity. 

This  generalization  of  the  Wishart  distribution  is  particularly  well  suited 
for  our  purposes,  as  the  peculiarly  simple  form  in  which  principal  minors  of  V 
appear  allow  us  to  do  preposterior  as  well  as  prior  to  posterior  analysis  in  a 
very  simple  fashion, 

2,2   Extended  Natural  Conjugate  Distributions  of  (|i,  h) 

Using  (4)  allows  us  to  define  an  extended  natural  conjugate  distribution  of 
(ji,  h)  with  (r-1)  more  parameters  than  the  Normal-Wishart  density  when  the  data 
generating  process  is  (r  x  1),  giving  a  great  deal  more  flexibility  in  the  assign- 
ment  of  priors  to  {^,    h) . 
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(v) 
We  define  a  type  1  extended  Normal-Wishart  distribution  f  jgy^Cii^h  |m,V,n^y^)  as 

equal  to 

^'^^ttlE.  h  n)  f^[^h|V,  V) 

-V,   _         -V, 


-ir(Vj.+r-l) 


r*(v) 


(5) 


If  (5)  is  to  be  a  proper  density  function  V  must  be  PDS,  v  >  0,  and  n  >  0.   In 


add 


ition,  the  v.  must  be  small  enough  so  that  a.  >^(j-l),  j=2^...,r.   Otherwise, 


the  V.  are  unrestricted  in  sign. 


We  obtain  the  marginal  prior  of  |i  by  integrating  (5)  with  respect  to  h. 
If  a.  >  ^(j-1)  for  j=l,2,...,r,  n  >  0,  v  >  0,  and  y  is  PDS  then 

B  "—  (  V  +r  ^ 


(6a) 


V  . 


.  jij  (2Vj-j)  +  (m-ii)^J^   Hj(E-tk)^^^)   ^ 
where  H^  s  (2v.-j)n  V^J^    and  H^  =  (Vj.-2p)n  v'-*-.   When  H^  and  all  H., 
j=l,2, . . . ,r-l,  are  PDS,  then 

D(ji|in,V,v)  CC  fs^''^ii|E.iv^V^-23)  jlj  fs''\H.^^^|mfJ^Hj,2Vj-j)     (6b) 

where  fg   (^i '■■'■' jm^-' ■' ,  H.,  2v.-j)  is  the  non-degenerate  multivariate  Student 
distribution.  '    i . 


Proof:   Integrating  over  R,  we  have 

h 


D(^|m,V,n,v)   Te-itr  h{n(j^-m)(^-tn)t+V)  |h|^'''^      i^  \^[j]\"^ 


J-1 


dh 


The  integrand  above  is  the  kernel  of  an  extended  Type  1  Wishart  density  with 
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parameter  ( n ( m-^)  ( m-ji)  +V^  v)  where  v  =  (Vj^,V2, . .  •  ,Vj._j^,Vj.+5)  .   Hence  defining 

we  have  by  use  of  (4a)  , 

-a, -^6     1  r ■ 1  "^ • 

D(ii|m,V,n,v)  <X|n(y^-m)(ia-m)'^+v|        •  ^.U^    |  [n(ii-in)(y^-m)''+v'-^  ^  |  :  J  " 

=  (1  +  (ix-m)^nv"h(ii-m))   ^  ^  '  JhJ  (I+C^-E)^^^  ("V^^^   )(li-IB/-^^)   ^ 

Provided  that  V  is  PDS ,  v  -2p  >  0,  n  >  0,  and  2v.-j  >  0,  j=l,2, . . . ,r-l, 

r  il"-'-  -1 

H.  =  n(2v.-j)V'--^-'    is  PDS,  j=l,2, . . .  ,r-l,  H  =  (v  -2p)nV   is  PDS,  6=1,  and  we 

have  (6b).   If  V  is  PDS  and  n  >  0,  (6a)  holds  even  when  one  or  more  2v.-j  <  0 

as  long  as  a,  >  ^(j-l),    j=l,2,...,r. 

If  one  or  more  of  the  conditions  v  >  0,  V  PDS,  a.  >  ^(j-1),  j=l,2,..,,r  are 
violated,  the  marginal  distribution  of  jj  does  not  exist. 

The  constant  that  normalizes  (6a)  is 

S(V,v)  =  k(v)  2^|v|-2  {^n^  B(i,  a.-i(j-l))}-^  (6c) 

where 

k(v)  =  (Vj,-2p)       -Ui    (2V.-3)    ^  (6d) 


Proof:   Since  V  is  PDS,  there  exists  an  upper  triangular  matrix  T  such  that 
I^^T  =  V"''".   It  is  also  true  that  x'^^-'  I'^J-l^^tJJ    for  j=l,2, . .  .  ,r-l . 

Let  ^'-   =!'••' (ji-ra)'^-'-'  and  make  the  integrand  transformation  from  ^^  to  y  in 
(6a).   Since  the  Jacobian  of  the  transformation  is  Ivl^, 

D(^|v)  =  \l\^   k(v)  (l+/y)   ^'^  ^n}  (l+x^J^''ytJJ)'''j  .  (6e) 

Integrating  over  R  s  (y|"»  <  y  *"  "^i  yields  the  normalizing  constant.   First  we 


evaluate 


/(I  +z'l)"'"*  %   (1  +yf^^V^^)''^  d^  (6f) 


S 
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'lUU       "TW)  T 

^       2  '^1*2  r-1  _     J   2  "^ 


ax 
r 


2 
Make  the  integrand  transform  y,=x,    in  (6e)  and  write  it  as 

00        00  1 

0     0  (6g; 

Integrating  first  with  respect  to  x  ,  we  find  that  the  above  integral  equals 

00  00 

-r         r  r         r-l    "^2  r-2     J    "'^1   r-1   -^ 

2   B(i,apj  ...j  (l+i^li  xj^)    [.n^  (l+_j£ix^)   JliSi  X.2]  dx^...dx^_^ 

0        6 

Continuing  in  a  similar  fashion  yields  (6f)  as  equal  to 

2"'^  B(^,a^)  B(i,a24)...B(i,aj-i(j-l))...B(^,a^4(r-l))  , 
and  (6c)  follows  directly. 

Later  we  will  need  the  following 

~( 1)     ~(j)     ~(ri) 
Theorem  1:   Let  x   , . . . ,x   > • • • >x    be  (r  x  1)  mutually  independent  random 

vectors  which,  conditional  on  h=h,  are  distributed  according  to 

(r)   I—  ~ 

^M  ^'  \—>^'^0  >    "•  >  0,  j=l,2,.,.,nj  let  h  have  an  extended  Type  1  Wishart 

(r)   I  ~ 

distribution  fi)i  (blY^v^)  ^nd  define  ^  as  an  (r  x  1)  linear  combination  of 

the  xs, 

y  =  a  +  E  p.  x^-^^  , 

where  p.'s  are  constants.  Then,  provided  that  v  >  0,  a.  >  |-(j-l), 

j=l,2,...,r,  and  ^  is  PDS,  the  distribution  of  y  is  of  the  form  (6a) 

_  t    -1    -  P4(v  +r) 
D(y)<X.(l  +  (2-Z)   (k^  )iri>) 


where 


k  =  (E  ^) 


J 
If,  in  addition  v  -28  >  0  and  2\i.-\   >  0, 

}  J.    r  2  —        ' 
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D(z)*  fg^'^^zlz^  h\   v^-2p)  JnJ  fs^J\z^^^|i^J^  k^fj^  ,  2v.-j)  . 

Proof;   Since,  conditional  on  h=h,  the  x  s  are  each  independent  (r  x  1)  Normal 
vectors, 

where  2     , 

-  -/-  ^  P-  "■'■ 

y  =  a  +  Z  p.  x'--^''   and   k  =  (E  -i) 

^       —  J—  ^n. 

Integrating  with  respect  to  h  gives 

D(y)  -f^ldll,   hk)  4j^h|v,  V)dh 


% 


j^2  iy[j] 


The  integrand  above  is  the  kernel  of  an  extended  Type  1  Wishart  density  with 
parameter  (kC^-y)  C^-^)  +V,  v^)  where  y^  =  (v,  ,...  ,Vj._-|^,Vj.+l)  .   Formula  (7)  follows 
directly. 

The  marginal  prior  on  h  is  obtained  by  integrating  (5)  with  respect  to  ^. 
If  n  >  0,  Y  is  PDS,  and  a.  >  |-(j-l),  then 

D(h|m,V,n,v)  =  f,ii^h|v,  V )  cc  e's'^'^  ^  Y  |h|^''^    .U^    \h^^^\    "^'^   (7) 

Proof;      Integrate    (5)    over   -«  <  ji  <  +»: 

D(h|m,    Y,    n,   V)    =    J    f^^'^liliE^   hn)    f,$[^h|y,   v)dix 

R 

ii 

R 

If  a    type    1    extended    Normal-Wishart  distribution  with  parameter 
(m' ,    Y' ,   n',   v^')    is   assigned   to    (^,   h)    and   if   a   sample   then  yields   a   statistic 
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(m,  V,  n,  V  )  the  posterior  distribution  of  (^,   h)  will  be  type  1  extended 

Normal-Wishart  with  parameter  (m",  V*",  n",  v")  where 

fl        n"  >  0 
n"  =  n'+n   ,   6"  =  \     if  ,  m"  =  n""  (n'm'+n  m)  ,       (8a) 

0       n"  =  0 


(8b) 


r^j-1 


V"  =  v'  +v+r-f€i-f«'-6"-0-l   ,  v'.'  =  v'.   ,   1  <  j  <  r-1   , 
rr  JJ 

d'    +  V  +  n'tn'm'*^  +  n  m  m*^  -  n'V'm"*^  =  V"  if  y"  is  PDS 
V*"=  y  ~  ~  .         (8c) 

L  0  otherwise 

Proof:   As  in  the  proof  of  (7)  in  [1],  we  multiply  the  kernel  of  the  prior  density 

by  the  kernel  of  the  likelihood,  obtaining 

^4n'(tL-E')'h(^-m-)  i^jiS-  ^-itrhV-  ^^^H''     J   ,     ,-j-l 

-  '  ^'        ~^^^  .    (8d) 

.^4n(m-^)  h(m-ix)  |j^|i6  ^-^tr  h  V  j  ^|^(v+r-(t-l) 

r 
Noticing  that  this  differs  from  (7d)  of  [1]  only  by  the  term  .II2  \\^r  ■  ] 

which  is  left  unchanged  in  the  process  of  combining  the  kernel  of  the  prior 

density  with  the  kernel  of  the  likelihood,  we  may  use  the  results  of  the  algebra 

of  the  proof  of  (7)  in  [1]  to  give  the  posterior  kernel  as 

which  is  the  kernel  of  a n  extend  ed  type  1  Normal-Wishart  density  with  parameter 
(m",  V",  n",  V"). 

2.3  Mean  Vector  and  Variance-Covariance  Matrix  of  the  Marginal  Distribution  of  ^ 

When  (^i,  h)  has  a  extended  type  1  Normal-Wishart  distribution  with 
parameter  (m,  V,  v) ,  then 

eQ)    =  m   and   V(^)  =  E(h'-^)    .  (9a) 
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Furthermore, 

' — 1     -1   t 
E(h  )  =  T  3  I  (9b) 

where  T  is  an  (r  x  r)  upper  triangular  matrix  such  that  T  T=V   ,  and  t]  is  an 

(r  X  r)  diagonal  matrix  with  first  diagonal  element  t)^  =(a  -2)   ,  and  for 

J =2, J,. . .,r. 

Proof:   That  E(^)=m'  follows  from 

That  V(2)=E(K"^)  follows  from  formula  (24)  of  [  i]: 

VQ  =  Eh  \\hQ    +Vh^^|h(^)    • 
Since  E  iu(ji)=£}';  ^  constant  not  depending  on  h  and  since  V  i.  (ii)=h   ,  the  first 

x£  I  —  ~  xil  _      ~ 

~-l 

term  above  is  E(h  )  and  the  second  is  the  null  matrix  0. 

To  prove  (9b)  and  (9c),  let  ^   =1   (ti-m) '^ -^  ■'  where  T  is  the  upper  triangular 
matrix  such  that  T  T   =V,    and  make  an  integrand  transform  f rom  jj  to  y  in  (t^), 


giving  as  in  (6e) 

D(y|v)  =  S(I,  V)  (1  +2%)   ^  ^  jtj  (1  +  ytJ^'x^-"^)  "^ 


a,-4  _  1        r  -it  r  .1  -V. 


Then 

1^^  =  vGi)    =   E(y^)  =    J     vl   D(l|v)d2 

2 
or  letting  x.=y.,  i=l,2,...,r,  and  making  this  integrand  transform  in  the  above 


integral, 


00 


V(y^)  =  2   S(I,  v)J  ...  j   xf  [^^2  ^j^] 

•  (1  nil  V  ''^[js;  (^+ii  -/>  '^]dxi...dx^ 


0     0  1 

-^1-2, r-1     1     -v^ 
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=  S(I,  V)  [2"'^  B(i,ap  B(i,a24)..-B(i,a^_^-i(r-2))  B(^,a^4(r+1))  ] 


2(a  -2)"  V  -2 


Evaluating  E(y  )  for  i=2,3,...,r  in  a  similar  fashion  yields  (9c).   Then  (9b) 
follows  directly. 


2.4  Mean  Matrix  of  h  =a  and  Variance  of  a.. 
=    11 

We  have  E(h  )=V{^)    in  (9b).   Here  we  derive  the  variance  of  a  ,   Q:=l,2^...,r 
as  a  function  of  the  expected  values  of  products  y.y.y.y.  of  elements  of  ^  as 
defined  in  the  proof  of  (9c),  and  explicitly  evaluate  V(aii)  and  V(022)  fo'^  r=2. 
As  mentioned  earlier,  we  may  use  the  ideas  of  the  proof  of  (9c)  to  evaluate  all 
V(cr  p)  and  cov(a  „,  a   o)  terms,  and  will  do  so  in  a  later  draft. 

We  now  show  that  if  an  extended  Type  1  Normal-Wishart  distribution  with 
parameter  (m,  n,  V,  v^)  is  assigned  to  (ji,  h) , 

V(r^  =  V(?  )  =  4  .  S  .   t-l  t"^  t"^  t']   E(y.y.y,yj  -  E^(S     )  (10a) 

where  t  g  is  the  76th  element  of  the  upper  triangular  matrix  T,  T  T  =  V   ,  the 
sum  is  over  all  1  <  i,j,j,/  <  r  and  the  y.s  are  elements  of  y,  distributed  as  in 
(6e). 

Proof;   From  formula  (23)  in  Martin  [i],    E([h'  J  )  =  E(a   )  =  j  E(jl  ).   As  the 
trix  I  is  such  that  t"  ^  =  ti.  H^  =  j?l  t^j  y  j ,  a=l,2,...,r,  and  so 


ma 


leading  directly  to  (10a), 

Notice  that  the  density  of  ^  ^'^    (6e)  is  symmetric  about  0  and  the  elements  of 
^  are  uncorrelated.   Hence  only  terms  involving  all  even  powers  will  be  non-zero, 
i.e.  E(y^  y^),  Ui ,    and  E(yj)  . 
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When  r=2,  2   -1  4    2  ~ 

Ti,,(t,,)    E  (a,,) 


(a^-j)      (^2-2) 


and 


^<°22)  =  -^  t<'n)'  -  ^<'n''(42)'  r^  -  <'2^>'  l^  ■  r^l) 

^^    2(a„-4)  ^^    ^   "111  ^11   ^1  ^ 


2  2' 


Proof;   From  (10a)  we  have 


-  £^(0^2)   .  (10c) 


-4v     ,r     -1  ~  -.4.     -4   ~4 
and 


EQ  =  E([  .Z^  t-^^  y.]-)  =  t^^  eGI) 


^(.t,)    =  E([i,  t-^l  y.]S 


-14   ~4       -12    -12   ~-v^       -14   .~4v 
('2P   ^(^l)  +  2(t2j)^  (t22)   E(y^  y2)  +  (t22)   E(y2) 


We  may  evaluate  E(y.),  i=l,2,  and  E(y  y„)  directly,  using  the  same  ideas  of  the 


proof  of  (9b)  to  find 

~4  ■? 

E(y:)  -       ^ 


'   '  ^(^2-1)  (^2-1) 


^4         3(a2-l)(a2-2) 

E(y  )  =  r r-   , 

4(a^-l)(a^-2)(a2-|)(a2-f) 


E(yT  y.) 


(a2-l) 


^   2    4(a^-l)(a24)(a24> 


Using  the  fact  that 

V(a.  .)  =  ^  E(^^)  -  E^a.  .)   ,     i=l,2, 
the  above,  and  (9b),  we  have  (10b)  and  (10c). 
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2.5  An  Example 

To  illustrate  how  the  results  of  section  2.4  give  us  more  latitude  in 
assigning  a  prior  to  (^^  h)  we  treat  a  (2  x  2)  numerical  example,  first  using  a 
Normal-Wishart  prior  and  then  using  an  extended  Type  1  Normal-Wishart  prior. 
In  each  case  we  wish  to  assign  a  prior  such  that 


e(m) 


vQ  = 


11.0    5.5 
5.5   11.0 


E(o)  = 


110     55 
55    110 


V(o^P  =  605.0   ,     V(a22)  =  1210. 

After  observing  a  sample  yielding  statistics 

'l50.0 


n  =  10   ,   V  =  8   , 


m  = 


250.0 


v-2 


V  = 


220    20 
_  20    55_ 
we  also  wish  to  find  the  posterior  of  (^,  h)  in  both  cases. 


1360. 

120.0 


*%*    '\* 


2.5.1  Analysis  Using  Normal-Wishart  Prior 

We  may  match  EC^),  E(a),  and  v(^)    by  setting 

200 
200 


E(^)  -  m- 


E(g)  = 


1 


v'-2  = 


V 


110     55 
55    110 


vQ  -  ~.    E(o)  = 


120.0 
330.0 


11.0     5.5 

_5.5    11.0 

so  that  n'=10.   We  must,  however  use  V(a^^)  and  V(a„^)  to  determine  v'  and  V' , 
and  as 
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2  -x*  2.  e'^ 

M(l^     ^   ^^  ^""ll^    24,200  ,   „,~  ,  ^^  ^^22^   24,200 

V(a   )  =  ;— T =  — f— T—  apd   V(a,9)  =  ;— 7 =  — Ft— 

11      V  -4      V  -4  ^l  V  -4      V  -4 


44   ,        V' 


when  a  Normal-Wishart  prior  is  assigned  to  (^,  h) ,  it  is  impossible  to  satisfy 
both  prior  specifications  V(a,,)  =  605.0  and  7(022)  ~  1210.0.  We  arbitrarily 
choose  the  former^  yielding 

'~4620   2310 
_2310   4620 
Once  the  sample  has  been  observed  formula  (7)  of  [1]  gives  posterior 
parameters 

n"  =  n'+  n  :=  10  +  10  =  20  , 

V"  =  v'+  V  +  r  +  5  +  6'-  6"-  <t'  -  1  =  44  +  10  -  1  =  53  , 
n  =  n'n/n"  =  (10)(10)/20  =  5   ,    (redundant)  , 

t 


V"    =   V'+  V  +  n    (m'-m)(m'-m) 

'l75 


18^480  -10,070 

•10,070  17,450 


m"   =  n"    \n'ra'+  n  m)    = 


225 


Posterior  to  the  sample  we  have 

175 


E(ii) 


E(o) 


225 


v"-2  = 


V" 


362.4   -197.5 
■197.5    342.2 


and 


V(u)  =  ^n  E(o) 


18.12    -9.875 
-9.875   17.11 


2E  (a   ) 

V(a  )  -.   n-J^  =  5361, 

11      V  -4         ' 


2E  (022) 
^<^22^  =   (v"-4)   =  "^7^°- 
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Notice  that  the  sample  has  drastically  increased  the  variances  of  2,  of  a^^ 
and  of  o«„.   This  occurs  because  the  sample  mean  vector  is  very  far  from  m' 
when  measured  in  units  of  the  standard  deviation  of  ^     and  ^  as  specified  in 
the  prior. 

2.5.2  Analysis  Using  Extended  Type  1  Normal-Wishart  Prior 

If  we  use  the  extended  natural  conjugate  prior  family^  we  may  match  E(^), 

E(g)j  yQ.) ,    V(a. ,),  and  7(022)-   ^°  ^^"^  ""^^  parameters  (m' ,  v' ,  n',  v ' )  in  this 
case,  we  note  that 

'200 
200 


eQ    =  m' 


and 


E(a)  =  T'"-^T]  T''^ 


V(^)  =  n'E(o)  = 


110    55 
_  55   110 

11.0     5.5 
5.5    11.0 


2E  (a   ) 
^(^iP  =   v'-4   =  ^°^-° 


V(a22) 


..l^/,..l^     21  ''   ^^'-''21  ''  ^^2  ''  r]^^ 


4(a^-^)(a'~) 


a'-2 


4-  it-V   ^  .  !^}  -  E\a^^)   =  1210.    . 

^11  ^i 


Solving  the  above  formulae  yields 
■^200 

200 


m'  = 


n'  =  10 


v'  =  (44,  13.23) 
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,,-1 


67.97      0 
33.99    36.51 


.02381 
0 


0 
.06179 


V'  =  !''■'"  T'*" 


4620    2310 
2310    2488 

After  observing  the  sample^  we  may  compute  the  posterior  parameters; 

n"  =  n'+  n  =  10  +  10  =  20  , 

V^'  =  V^+v+r-6+6'-  &"-  <D  -  1  =  44  +  10  -  1  =  53  , 

v!^  =  v'  =  13.23  , 

n  =  n'n/n"  =  (10)(10)/20  =  5  ,    (redundant)  , 


V"  =  V'+  V  +  n  (m'-m)(m'-m) 


18,480 
-10,070 


•10,070 
15,318 


m"  =  n""-^(n'm'  +  nm)  = 


175 


225 


From  these  parameters  we  may  compute 


n..-l 


135.8       0 
-74.15    99.10 


Ti"  = 


.01961      0 
0     .03992 


leading  to 


E(a)  =  T"'-"-  3"  T"^ 


362.4     -197.5 
■197.5      499.8 


I 
n' 


V(ii)  =  -.,  E(g)  = 


V(a,P 


^2  2 


181.2   -98.75 
-98.75  249.9 

=  5361. 


'..O'. 


and 

In  this  case,  the  posterior  variance  of  022  is  drastically  larger  than 
the  prior  variance  of  0^22  ^nd  also  much  larger  than  that  of  a„„  when  a  Normal- 
Wishart  prior  was  assigned. 
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3.   Sampling  Distributions  with  Fixed  n 

We  assume  that  a  sample  of  size  n  is  to  be  drawn  from  an  r-dimensional 
independent  Multinormal  process  whose  parameter  (^,  h)  is  a  random  variable  having 
an  extended  Type  1  Normal-Wishart  distribution  with  parameter  (m' ,  V' ,  n',  v'). 


■"N^      "Nrf  I 


3.1  Conditional  Joint  Distribution  of  (m,  y|^,  h) 

The  conditional  joint  distribution  of  the  statistic  (m,  V)  defined  in  formula 
(8  )  of  [1]  given  that  the  process  parameter  has  value  (^,  h)  is,  provided  v  >  0, 

D(in,  V\^,    h,    V)  =  4^\mlii,  hn)  f^^\v|h,  v) 


r\rf    i^t 


3.2  Unconditional  Joint  Distribution  of  (m,  V) 

The  unconditional  (with  respect  to  (^,    h))  joint  distribution  of  (m,  V)  when 
(li;  b)  has  been  assigned  an  extended  Type  1  Normal-Wishart  prior  with  parameter 
(£*  ^  YS  '^' }    v')  has  density 

D(m,  V|in*,  V',  n',  v'j  n,  v) 


=  J  Jf^^^(m|^,  hn)  f^^'^Vlh,  V)  f^liii,    h|m',v',n',v')di,  dh 


(11a) 


R  R, 

a.    h 

where  the  domain  of  integration  of  y^^  is  R  =  (-»,  4w)  and  of  h  is  R 
It  follows  that  if  v'  >  0,  n'  >  0,  V'  is  PDS ,  and  a!  >  •g-(j-l), 

D(E.  V|m',  V',  n',  v';  n,  v)co|v|^^"^  |b | P ' "iCV'+r-l) .rgl  |b[j]|"^j   (lib) 


where 


1=  n^(m-m')(m-m')''  +  V'  +  y   ,  P '  =  jll  v'   •' 
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Proof;   Following  the  initial  steps  in  the  proof  of  formula  (10)  in  [1],  we  may 
write,  letting  n  =n'n/n", 

D(m,  Y|m',  y' ,  n',  v ' J  n,  v) 

h 

This  integral  is  proportional  to 

1     p    -itr  h{n  (m-m')(m-m')''+V+v')    ^v"  .  ^  "V-  i 


h 


Letting 


B  =  n^(m-m')(m-m')^  +  Y  +  V' 


the  above  integral  may  be  written  as 

h 
The  integrand  is  the  kernel  of  an  extended  Type  1  Wishart  density  with  parameter 

(B,  V^")  where  v"  =  (v-!  , .  •  .  ^V '  ■,  ,    v")  .   Hence  apart  from  a  normalizing  constant 
depending  on  neither  V  nor  B, 

.  D(in,V|in',y',n',v'jn,v)  QC  |V|2^  ^  III       ^  jSzIl    I   ^   • 

3.3   Unconditional  Distribution  of  m 


When  n  >  0,  a  1  >  ^(j-1),  v'  >  0,  and  v'  is  PDS , 

D(m|m',  V',  n',  v ' j  n,  v)  (12a) 

^   1      P'-i(v;+r)  „  ,  r  •i'^  r  -f^     rn"^' 

OC   (1  +  n^(m-m')'^V'   (m-m))  •   ^J  (l+n^(m-m' )  ^  ^  J  V^J-"  (m-my-'-')  J 

so  that  if,  in  addition,  2v'.-j  >  0,  j=2,3, . . .  ,r-l, 
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D(m|m',  V',  v'j  n,  v) 

fs^'^^ElE'.  H^.  v^)  jhJ  fs^^E^^^lE'^^^  Hj.  2v!-j)         (12b) 

where  H^  £  (v^-2p' )n^V' '-^  and  H^  =  (2v  !- j)n^V' ^  J  ^   ,  j=l  ,2, . .  .  ,r-l.   Both  (12a)  and 
(12b)  hold  even  if  v  <  0. 


Proof;   The  kernel  of  the  marginal  likelihood  (4b')  in  [1]  of  m  given  the  parameter 
(,j£,  h)  and  that  n  >  0  is  Normal  with  parameter  (ji,  hn) .   Also,  the  kernel  of  the 
prior  of  ^   given  h=h  is  Normal  with  parameter  (m,  hn  ) . 

Since  ^  and  £  =  m-jj  are,  given  h=h,  each  independent  Normal  random  vectors 
and  since  a  sum  of  independent  (r  x  1)  Normal  random  vectors  is  (r  x  1)  Normal, 
it  follows  that  n;=^+6  is  Normal  with  mean  vector 

E(m)  =  eQ    +   E(e)  =  E(^)  =  m' 
and  variance-covariance  matrix 

V(m)  =  V('^)  +  V(|)  =  h'^(i,+  ^)  =  (hn^)"^   . 
Hence  the  distribution  of  m  unconditional  as  regards  ji  but  conditional  upon  h=h 
is  Normal  with  parameter  (m',  hn  ), 

To  find  the  distribution  of  m  unconditional  as  regards  h  as  well  as  ji,  we 
note  that  the  marginal  distribution  of  h  is  from  (7)  fiji  (h|v',  v')  so 

D(m|m',V',n',v';n,v)  =  f  ^^''hEllB' >    hn^)  fwi^^=l^''  ^'^^^ 

Since  n  >  0  and  n'  >  0  here,  5=1  and  the  above  is  proportional  to 

n    -in  (m-m')''h(m-m')    i    -^tx   h  V'    h'-l      ^  -y' 

/  e  '"--=--   |h|   •  e     "  ~   |h|   '   jS2  l^[j]l      ^^ 


h 


4tr   h(n^(m-m')(m-m')''+V'}  ^(v^-Dr  -v\.-^ 

lul  TT  IV,  I  J 

R 


e  '  |h|  .22    Ih^jjl  dh      . 
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Observing  that  the  integrand  in  the  integral  immediately  above  is  the  kernel  of 
an  extended  Type  1  Wishart  distribution  with  parameter  (n  (m-m' ) (m-m' )  +V',  v  ) 
where  v  =(v|, . . . ,v' _, ,v'+l) .  Both  (12a)  and  (12b)  follow  directly. 

3.4  Unconditional  Distribution  of  V  When  v  >  0 


When  V  >  0,  v'  >  0,  and  v'  is  PDS ,  the  kernel  of  the  distribution  of  V 
unconditional  as  regards  m,  ^,    and  h  is 
D(y|E',  V',  n',  v'j  n,  v) 

^  .3.+i(v"+r)-l    jSlU^  +  HJ    I 

|v+v'|      "^ 

Proof;   From  (4c)  of  [1]  the  marginal  likelihood  of  V  is  proportional  to 
l^jlv-l  g4tr  h  V  |^|i(v+r-l) 

Multiplying  this  kernel  by  the  kernel  of  the  prior  distribution  (5)  of  (^,   h) 
with  parameter  (m' ,  V' ,  n',  v')  and  integrating  over  the  range  of  ^  gives ^  when 
n'  >  0, 

D(V,  h|m',  V',  n',  v'>  n,  v) 

iv-1  ,   >(v;!-i)-i    -^, 


^(V"-1)-1   4tr  h(V+V') 
CC    |V|2''^  Ihf  ""  e 


Integrating  the  above  kernel  over  R,  ,  we  have 
D(V|m',  V',  n',  v';  n,  v) 

iv-1  n     -itr  h(y+y')   i(v;:-i)-i  ^  ,    -v\_^ 

OC  |V|     j   e  |h|    "      .112  l!5[j]l      dh   . 

The  integrand  in  the  above  integral  is  the  kernel  of  an  extended  Type  1  Wishart 
density  with  parameter  (Y+V',  v^)  where  v=(v.[, ...  ,v'  ,),  v"-l).   Formula  (13) 
follows  directly. 


-  24  - 

4.   Preposterior  Analysis  with  Fixed  n  >  0 

We  assume  that  a  sample  of  fixed  size  n  >  0  is  to  be  drawn  from  an 
r-dimensional  Multinormal  process  with  mean  vector  ^  and  matrix  precision  h 
are  not  known  with  certainty  but  are  regarded  as  random  variables  (ji,  h)  having 
a  extended  Type  1  Normal-Wishart  prior  distribution  with  parameter  (m,  V',  n'^  v') 
where  n'  >  0,  v '  >  0,  a !  >  ^(j-l)j  however,  V'  may  or  may  not  be  singular. 

4.1  Joint  Distribution  of  (m",  y") 

The  joint  density  of  (m",  f)    provided  n'  >  0,  v'  >  0,  v  >  0,  and  a'  >  |-(j-l) 

is, 

D(m",  f'\m',    Y' ,    n' ,    v ' }  n,  v) 


V"-V'-n*(m"-m')(m"-m')"  2-  _^, 

OL   -^-^ "■  7.  ■■"  7. JnJ  If  tJJ|    J 


TTTTrr 


^„|-p'-f^(v"+r-l) 


where 


n*  =  n'n"/n  (14b) 

and  the  range  of  (m",  V")  is 

R/-  "  w<\   =  Ue'SV")|-»  <  m"  <  4xo,  and  y"-V' -n*(ra"-ra' )  (m"-m' )  *^  is  PDS }  . 

Proof:   From  formulas  (7),  (18a)  and  (18b)  of  [1],  we  have 

(m,  V)  =  (i(n"m"-n'm')  ,  V"-V' -n*(m"-m' )  (m"-m' ) '^)    ,  (15) 

and  from  the  proof  of  (17)  in  [1],  the  fact  that  J(m",  y"j  m,  V),  the  Jacobian 
of  the  integrand  transformation  from  (m,  V)  to  (m",  V"),  is  a  constant  involving 
neither  m,  nor  V. 

Making  the  transformation  (15)  in  (11),  since  the  Jacobian  is  just  a  constant 
of  proportionality,  we  obtain  (14a). 
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As  in  (17a)  of  [1],  this  density  exists  only  if  v  >  0,  since  the  numerator 
in  the  first  term  above  vanishes  when  v  <  0.   The  kernel  exists,  however  even  if 
y'  is  singular  provided  that  v  >  0. 

4.2  Some  Distribution  of  m" 


The  distribution  of  m"  unconditional  as  regards  ^i  and  h  provided  n  >  0, 
v'  >  0,  a!  >  ^(j-1),  j=l,2,...,r-l,  and  V'  is  PDS ,  is 


D(m"|m',  Y',  n' ,  v ' >  n,  v) 

P  ~ 
r     -_  _  -  =v  —  -  • 

■V 


(v'-2p'+(m"-m')'^H  (m"-m')      "^  (16a) 

.  JnJ  (2v^-j)+(m"[JJ.m'tJ])tH°(rn-tJJ.m'fJ]))"''^ 


where 


If  in  addition,  2v'.-j  >  0,  j=l  ,2, . . .  ,r-l,  and  v'-2|3'  >  0,  we  may  write 
D(m"|m',  Y',  n',  v'>  n,  v) 

^    fs^^^m"|m',  E°,    v;-2p')  ^nj  fg^  J\m"  ^  ^  ^  |  m' ^  J  ^  fy    2v'-j)   . 
Proof:   The  distribution  of  m  unconditional  as  regards  jj.  and  h  when  (n,  h)  is 
assigned  an  extended  Type  1  Normal-Wishart  prior  is  of  the  form  (12)  with 
parameter  (m' ,  y',  n',  v ' ;  n,  v) .   By  Theorem  1  proven  in  substitution  2.2,  since 

m"  =  1,  (n'm'  +  nm)   , 
—    n"    —     — 

the  distribution  of  m"  is  of  the  form  (12)  with  parameter  (m' ,  (-„)  v' ,    n',v'jn,  v) 
Formula  (16)  follows  by  substitution  of  (-„)^V'  for  V'  in  (12). 


The  conditional  distribution  of  m"  given  V"=V"  provided  v  >  0,  n'  >  0,  v '  >  0, 
and  a'.   >'^(j-l),  j=l  ,2, . . .  ,r-l ,  is 


(r).„.. 


D(ra"|m',  V',  n',  v ' ;  n,  v,  V")  =  f-g  (E"Ie'^  »*>  v)  (16a) 
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where 

H^  =  vn*(V"-V')'-'-  .  (16b) 

Proof;   The  kernel  of  the  conditional  distribution  of  m"  given  y"=V"  is 
proportional  to  (14a),  so 

D(m"|m',  V',  n',  v' }  n,  v,  V") 

OC  |Y"-y'-n*(m"-m')(m"-m")'^|2^"'^ 
Hence,  as  shown  in  the  proof  of  (20)  in  [1],  this  distribution  of  m  is  inverted 
Student  with  parameter  (m' ,  H^,  v) ,  and  (y"-V')"   is  PDS  as  long  as  v  >  0. 
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